docs: CONTRIBUTING + ideation-lead wording cleanup (#95 / #82 #89 follow-up)#101
Conversation
#95 / #82) Wording-only follow-up to PR #82's CONTRIBUTING semantic-scope refresh. Two minor fixes that drifted as the test layout matured: - Bug-fix checklist now points to `tests/` (fixture / e2e / boilerplate lint) for regression placement, with `.github/workflows/ci.yml` called out as wiring rather than the test home. - LESSONS.md path is given as the full `plugins/preview-forge/memory/` location to match §1 LESSON guidance higher in the file. No change to the 57-check verify count, 144 agent total, 14 commands, 26 advocates, or commit-prefix examples — all of those still match the filesystem snapshot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
#95 / #89) Wording follow-up to PR #98's A-4 enforcement section. Two spots in `agents/ideation/ideation-lead.md` still described the cascade with the older conceptual labels only; this commit makes the script-canonical mode names visible everywhere they appear: - §1 cascade table: each tier row now annotates its `filled-ratio-gate.sh mode=` value (`ground-truth` / `hint` / `low-confidence` / `fallback-omit-spec`) next to the conceptual high/medium/low/fallback label, and the fallback row spells out that the IDEA_SPEC_CONFIDENCE line is dropped (not just the IDEA_SPEC). - §2 advocate-dispatch template: replaces the old "fallback인 경우 라인을 빼라" LLM-discretion prose with a script-enforced contract — I_LEAD derives the IDEA_SPEC / IDEA_SPEC_CONFIDENCE *structure* from `filled-ratio-gate.sh --prompt-fragment` (which lines, which confidence label, whether the confidence line is omitted) and only substitutes the actual `idea.spec.json` payload into the `<splice runs/<id>/idea.spec.json …>` placeholder. Advocates consume whatever shape arrives without second-guessing tier or omit decisions. This closes the LLM-trust → script-enforced loop for A-4: the gate is now load-bearing not only for tier classification but also for the advocate-prompt construction itself. No code or test changes; the byte-equal fixtures in `tests/fixtures/filled-ratio-gating/verify.sh` still pass and `tests/test-advocate-boilerplate.sh` (which lints advocates only, not ideation-lead.md) is untouched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 32 minutes and 23 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the contribution guidelines for bug fixes and refines the documentation for the ideation-lead agent. Key changes include specifying the location for regression tests and LESSON entries, as well as detailing the integration between the ideation-lead agent and the filled-ratio-gate.sh script for prompt scaffold generation. Review feedback identifies potential prompt pollution by placing internal instructions inside a template block, inconsistencies between the documented template and the script's actual output, and a failure to explicitly state that placeholders must be replaced with runtime values.
| # IDEA_SPEC + IDEA_SPEC_CONFIDENCE 두 줄의 **구조**(어느 라인을 포함할지, | ||
| # 어떤 confidence 라벨을 쓸지, fallback에서 IDEA_SPEC_CONFIDENCE를 누락할지)는 | ||
| # I_LEAD가 §1 Enforcement 단계에서 | ||
| # `bash scripts/filled-ratio-gate.sh --prompt-fragment runs/<id>/idea.spec.json` | ||
| # 의 stdout으로 결정한다 (byte-stable, 4-tier별로 정해진 scaffold). I_LEAD는 그 | ||
| # scaffold의 `<splice runs/<id>/idea.spec.json …>` 자리에만 실제 JSON 본문을 | ||
| # 끼워넣고 그 외 wording·라인 유무는 손대지 않는다 — tier 판단·omit 여부 등은 | ||
| # advocate 쪽에서 second-guess 하지 않는다. fallback-omit-spec tier에서는 | ||
| # IDEA_SPEC_CONFIDENCE 라인이 의도적으로 누락되어 들어오므로, advocate는 그 | ||
| # 부재 자체를 "spec 없음" 신호로 해석하면 된다 (별도의 default 값을 만들어내지 말 것). |
There was a problem hiding this comment.
The new explanation block correctly describes the contract with scripts/filled-ratio-gate.sh, but its placement and the template structure immediately following it present a few issues:
- Template Inconsistency: The template lines (133 and 141) are inconsistent with the script's canonical output. The script outputs
IDEA_SPEC_CONFIDENCEbeforeIDEA_SPEC(seefilled-ratio-gate.shlines 106-112), and uses different placeholder wording (verbatim — ground truth). Since the agent is forbidden from touching the scaffold wording (lines 129-130), this discrepancy may cause confusion. - Prompt Pollution: These instructions are placed inside the
textcode block intended as the prompt template. This may lead theI_LEADagent to include these internal instructions in the final prompt sent to the Advocate. - Redundancy: Having individual lines for
IDEA_SPECandIDEA_SPEC_CONFIDENCEin the template is redundant now that the agent is instructed to use the script-generated scaffold. - Placeholder Instructions: Per repository guidelines, when documenting instructions for agents that involve placeholders (e.g.,
<...>notation), you must explicitly state that these placeholders must be replaced with actual runtime values to prevent them from being interpreted as literal strings.
Consider moving these instructions outside the code block and updating the template (lines 133, 141) to use a single placeholder that matches the script's output order and wording, while ensuring the requirement to replace placeholders with runtime values is explicitly stated.
References
- When documenting instructions for agents that involve generating files with placeholders (e.g., <...> notation), explicitly state that these placeholders must be replaced with actual runtime values to prevent them from being interpreted as literal strings.
1 participant
Summary
Pure-doc cleanup cluster from #95, finishing the wording follow-ups deferred from PRs #82 and #89.
tests/(fixture / e2e / boilerplate lint) instead of the stale "in.github/workflows/ci.ymlif hook-related" path, and the LESSONS.md path is fully qualified to match §1. All other counts on the page (57 verify checks, 144 agents, 14 commands, 26 advocates) re-verified against the filesystem and left as-is.agents/ideation/ideation-lead.mdA-4 alignment: the §1 cascade table now annotates each tier row with its script-canonicalfilled-ratio-gate.sh mode=value (ground-truth/hint/low-confidence/fallback-omit-spec), and the §2 advocate-dispatch template replaces the old "fallback인 경우 라인을 빼라" LLM-discretion prose with a script-enforced contract — I_LEAD derives the IDEA_SPEC / IDEA_SPEC_CONFIDENCE structure (which lines, which label, whether to omit) fromfilled-ratio-gate.sh --prompt-fragmentand only substitutes the realidea.spec.jsonpayload into the<splice runs/<id>/idea.spec.json …>placeholder.Side-effect / breaking-change assessment
tests/test-advocate-boilerplate.shP*.mdonly, notideation-lead.mdscripts/verify-plugin.shideation-lead.md(confirmed via grep)tests/fixtures/filled-ratio-gating/verify.shfilled-ratio-gate.sh; no dependency onideation-lead.mdtexttests/fixtures/security/verify-security.shtests/e2e/mock-bootstrap.sh {standard,pro,max}No code or test changes. Wording-only.
Codex review
ONE pass against
origin/main. Caught 1 P1, applied:--prompt-fragmentstdout 그대로" — but the fragment uses<splice runs/<id>/idea.spec.json …>placeholders, so taking it literally would have stripped the real spec from non-fallback runs. Fixed to clarify I_LEAD uses the fragment for structure (line presence, confidence label, omit decision) and substitutes the actual JSON payload into the placeholder. Re-verified all regressions still pass.P2/P3: none.
Test plan
bash scripts/verify-plugin.sh→ Pass: 57 / Fail: 0bash tests/fixtures/security/verify-security.sh→ all Phase 1 defenses holdingbash tests/fixtures/filled-ratio-gating/verify.sh→ all 4-tier byte-equal assertions passbash tests/test-advocate-boilerplate.sh→ 26 advocates share single boilerplate hashbash tests/e2e/mock-bootstrap.sh standard→ 9 iframes, mode=ground-truthbash tests/e2e/mock-bootstrap.sh pro→ 18 iframes, mode=ground-truthbash tests/e2e/mock-bootstrap.sh max→ 26 iframes, mode=ground-truth, framework lint rc=0Closes part of #95 (#82 + #89 wording-fix sub-cluster).
🤖 Generated with Claude Code